1 link tagged with all of: mechanistic interpretability + linear probes + ai
Click any tag below to further narrow down your results
Links
This paper explores how large language models make decisions during reasoning. It demonstrates that these models often encode their choices before generating text, influencing their subsequent thought processes. The research shows that altering initial decisions can change reasoning outcomes significantly.
- A linear probe can decode whether a model will call a tool from its activations before it generates any reasoning text, sometimes before any tokens at all
- Artificially flipping this early "decision direction" causes the model to switch its tool-use behavior in 7% to 79% of cases depending on model/benchmark
- When steered toward a different decision, the model's subsequent reasoning rationalizes the new choice rather than resisting or correcting it, suggesting the "thinking" is post-hoc justification rather than genuine deliberation